Using the Information Center, there is a page about Getting Fixes. A product fix might be available to resolve your problem.

Is it running

Make sure the Event Broker is started, and if not, then start it. (See section Restarting the Event Broker below)
You can check if the correct processes are open and if the correct ports are open (see section Using netstat).

Checking on Windows

• Administrator Tools
   
  Services if you open that up from the Control Panel.

Warning If dataflowengine is restarting, then generally there is a problem with Exit, Class loading problem in exit, or a class path. Look for the PID changing every 30 seconds.. the PID will go up.

Checking on Linux

Check the local error log is - the syslog. Where syslog messages are sent depends on how you configure your UNIX or Linux system. Try the following command to see the last 50 lines of the linux syslog:



Look for the following processes to be running:

• bipservice.exe
• bipbroker.exe
• biphttplistener.exe
• DataFlowEngine.exe

Examples with grep, we can limit which results come back.:



You can kill a task with option -9, try the following if you want to end the process.

Restarting the Event Broker

See Starting and stopping a broker for full details.

You need to know the broker's name in order to run the start/stop commands. STA ships with BRKR_SCCS and CMGR_SCCS. Thus to restart you must first stop both and then start both.


To Stop the Event Broker

1. Run the install_dir/bin/mqsiprofile.cmd command to set up the environment for a single targeted runtime.

You must do this before you can run any of the WebSphere Message Broker commands.

To stop a broker, enter the following commands on the command line:

Note Sometimes broker BRKR_SCCS won't shut down, in that case manually end the DataFlowEngine task. On Windows, use the Task Manager to kill the task. On unix, first grep for the task then kill it by kill -9 <taskid> (see section on 'Is it running?' for Linux).



2. Stop IBM WebSphere MQ. On Windows, this means go to Windows Services and stop the IBM MQSeries service. Run the following command even if MQ service is running:



To Start the Event Broker

1. Start IBM WebSphere MQ. On Windows, this means go to Windows Services and start the IBM MQSeries service. Run the following command even if MQ service is running:

2. Run the install_dir/bin/mqsiprofile.cmd command to set up the environment for a single targeted runtime. You must do this before you can run any of the WebSphere® Message Broker commands.

Note: On windows, you can also run mqsistart/ mqsistop commands from Start - IBM WebSphere Message Brokers - Command Console.

Health checking and monitoring areas

Characteristics for message broker components - health-checking and monitoring areas

Broker
You can perform the basic health-checking of the broker environment by
ensuring that:

• All broker processes are running properly.
• Broker queue manager is running properly.
• There are no undesired error messages in syslog or event log.
• There are no error messages in WebSphere Message Broker stdout and stderr files.
• The database is running and there are no database related errors in either database log or the broker log.

Problems with the performance of a WebSphere Message Broker message flow
usually take one of two forms. The first is that the required message throughput
cannot be achieved. The second is that, although the required level of message
throughput is being achieved, the CPU cost is far greater than expected. When
the expected level of performance from a message flow is not met, it is usually
because of one of the following reasons:

• A lack of resources with which to run the message flow.
• A poorly configured environment.
• Poor response time from a dependent application.
• An inefficient processing algorithm for the message flow.
• Inefficient or excessive ESQL processing within the message flow.

Broker tuning considerations

The configuration of the message broker and the manner in which message
flows are assigned can have a noticeable effect on message throughput. You
can perform the following tuning options in a production environment:

• Trusted broker

To verify that traces are turned off, use mqsireporttrace command for all
execution groups (Example 7-2).

Verifying user trace and service tracing level

You can write automated scripts to query the tracing options for all the
brokers, execution groups, message flows, and the level of tracing (user level
and service level). Also, you should ensure that there are no trace nodes in
the processing paths of your message flows.

ODBC trace is turned off using the tracing tab of the ODBC function as
follows:

• On Windows, it is available in the Control Panel.
• On UNIX, it is controlled through the Trace keyword in the odbc.ini file. Possible values are 0 (off) and 1 (on), as shown in Example 7-3.
• On z/OS, it is controlled through the APPLTRACE keyword under the stanza [COMMON] in the BIPDSNAO file. Possible values are 0 (off) and 1 (on).

ODBC stanza on UNIX system




You should check for other product or system-related traces, such DB2 trace or operating system trace.

For more information, refer to the article Tuning a WebSphere MQ Integrator Broker, which is available at: Tuning a WebSphere MQ Integrator Broker

Where to look for jvm logs files

Windows

1. Go to Windows Services
2. Find "IBM WebSphere Message Broker component...", right click
    
   Properties
3. Go to Log On tab, check Local System account and Allow service to interact with desktop

Now when you right click to start the service, it will show up with dos window and you can see JVM messages!

Linux

JVM logs are located in /mqsi/ folder in files: standarderr.log and standardout.log

Where to look for event broker log files and change log levels

Local error logs Troubleshooting & support Using Logs Using Trace Changing user trace options

Restarting

You must restart the Event Broker for the logging to take affect.

Logging for Windows

The local error log is the Windows Event log (Application view or System view).

Logging for Linux

1) /var/log/messages

This is the primary area for determining problems with the event broker - it dumps a LOT of information into this file... a lot will be informational, but you may see some errors as well - any problems with the security exit (like a classpath issue), will be glaringly obvious.

2) /var/mqsi/components/BRKR_SCCS//stdout and stderr - java dump files... another place to look for errors

Note: If these two places don't point to the problem, then your next step is tracing...

Tracing

There are two main types of trace available in WebSphere® Event Broker: user trace and service trace.Typically, you utilize user trace for debugging your applications; you can trace brokers, execution groups, and deployed message flows.

Lets learn about the commands.

mqsichangetrace command
Changes a component's trace options.

Syntax (1): mqsichangetrace componentName -u -e egName -f messageFlow -l level -m mode -c size -r
Syntax (2): mqsichangetrace componentName -t -b -l level -m mode -c size

Command Options:
'-u' indicates user trace (may only be used with '-e')
'-t' indicates service trace.
'-b' denotes tracing of a component other than an execution group. -b is only used when -t is specified.
'-e egName' Name of the execution group
'-f messageFlow' Name of the message flow
'-l level' specifies the trace level, 'none', 'normal' or 'debug'
'-m mode' specifies the trace mode, 'safe' or 'fast'
'-c size' specifies the trace file size in kilobytes, default value of 4096 KB, maximum value is 2097151, which allows a trace file up to 2 GB (gigabyte) to be created.
'-r' specifies that the log should be reset.

The output for service and user trace generated by these commands is written to trace files in the log subdirectory. When you have completed the work you want to trace, use mqsireadlog to retrieve the log as an XML format file. Use either mqsiformatlog (to produce a formatted file) or an XML browser to view the XML records.

mqsireadlog command
BIP8114W: Retrieves the trace log.

Syntax (1): mqsireadlog componentName (-u | -t) -e executionGroupName -f -o outputFileName
Syntax (2): mqsireadlog componentName -t -b subcomponent -f -o outputFileName

Command Options:
'-u' indicates user trace (may only be used with '-e')
'-t' indicates service trace
'-b subcomponent' name of the command or subcomponent (for example, 'agent')
'-e executionGroupName' Name of the execution group
'-f' specifies that the log file is to be read directly from the file system (this flag is mandatory for service trace)
'-o outputFileName' file name to write the trace log.

Procedure to take a startup trace of a broker

First stop the broker and set the trace. There are a few ways to set the trace. One uses the mqsiservice command and the other uses the mqsichangetrace command.

# Stop the broker, in our case broker name is BRKR_SCCS

mqsistop BRKR_SCCS

# Enable broker agent trace either with option A or option B

# Option A

mqsiservice BRKR_SCCS -r Trace=debug

mqsiservice BRKR_SCCS -r TraceSize=2097151 

# Option B 

# use -u if you want to do user trace OR use -t if you want to do service trace

# both are explained above for the command mqsichangetrace

mqsichangetrace BRKR_SCCS -u -e default -l debug -m safe -c 2097151

mqsichangetrace BRKR_SCCS -t -b -l debug -m safe -c 2097151

Next, start the broker, recreate the issue and stop the broker.

# Start the broker.

mqsistart BRKR_SCCS 

# Recreate the problem (check for errors in the syslog or event viewer log) to capture the logs

# Stop the broker. Make sure the broker is stopped by checking ports 1506 &

# 1414, and checking executables Dataflowengine & bipservice to not be running

mqsistop BRKR_SCCS

Next read the log file and format it for better viewing. Find the results in trace.xml in XML format.

• In Windows. C:\Documents and Settings\All Users\Application Data\IBM\MQSI\common\log
• Unix. /var/mqsi/common/log

# Retrieve the trace log for the specified component. Use either option -t & -b, or -u & e.

# -u Read the log contents from the user trace log.

# -e Egroup is the label of the execution group for which log information is to be read

# -t Read the log contents from the service trace log.

# -b Read the contents of the log for the broker agent

mqsireadlog BRKR_SCCS -u -e default -o trace.xml

mqsireadlog BRKR_SCCS -t -b agent -f -o trace.xml

eg. mqsireadlog ConfigMgr -t -b agent -f -o trace.xml

eg. mqsireadlog BRKR_SCCS -e default -f -o trace.xml

# Format the XML log file to be more readable with the following command

mqsiformatlog -i trace.xml -o formattrace.log

Finally, turn off trace because there's additional processing for every activity in the component that you are tracing.

# Turn off trace.

mqsiservice <BrokerName> -r Trace=""

mqsiservice <BrokerName> -r TraceSize=""

mqsiservice <BrokerName> -r executionGroupTraceOverrideLevel=""

mqsiservice <BrokerName> -r executionGroupTraceOverrideSize=""

#To switch off Trace nodes in the default execution group, use the command:

mqsichangetrace WBRK_BROKER -n off -e default

If these steps don't point to the problem, then pick up the phone or IM someone on the MQ WMB team.

Security

How to turn on logging/tracing for security exit code
Change the logLevel line in exitsetting.ini (found in C:\Program Files\IBM\MQSI\6.0\bin) to FINE, FINER, FINEST, ALL, NONE, INFO, etc.
You cannot save the file until you stop the Broker services and the MQ services. Follow the instructions for Restarting the Event Broker. Once everything is stopped, then start up the Event Broker again.

Look for file RestImpl-0.log in folders:

• on windows, in the bin directory of the MQSI installation (c:\program gfiles\IBM\MQSI\6.0\bin)
• on linux, /var/mqsi

Enabling / Disabling Security Exit code

• Select Start > IBM WebSphere Message Broker 6.0 > Command Console to open the command console.
• In the command console, type the following commands:

Disabling Broker Security

# this will not crash our testing program which enters users, 

# but will simply allow all users to login

Enabling Broker Security:

mqsiservice BRKR_SCCS -r PubSubAccessControl=yes

mqsiservice BRKR_SCCS -r PubSubAuthorizationService=com.ibm.orgcollab.wbi.exits.DummyTrue

mqsiservice BRKR_SCCS -r PubSubPrincipalDirectory=com.ibm.orgcollab.wbi.exits.DummyTrue

OR

# this completely removes broker security

# clients will not be able to connect

mqsiservice BRKR_SCCS -r PubSubAccessControl=no

mqsiservice BRKR_SCCS -r PubSubAuthorizationService=

mqsiservice BRKR_SCCS -r PubSubPrincipalDirectory=Enabling Broker Security

mqsiservice BRKR_SCCS -r PubSubAccessControl=yes

mqsiservice BRKR_SCCS -r PubSubAuthorizationService=com.ibm.orgcollab.wbi.exits.rest.RestImpl

mqsiservice BRKR_SCCS -r PubSubPrincipalDirectory=com.ibm.orgcollab.wbi.exits.rest.RestImpl

Enabling / Disabling SSL

• Select Start > IBM WebSphere Message Broker 6.0 > Command Console to open the command console.
• In the command console, type the following commands:

Disabling SSL

mqsichangeproperties BRKR_SCCS -e default -o DynamicSubscriptionEngine -n clientAuthProtocols -v P 

mqsichangeproperties BRKR_SCCS -e default -o DynamicSubscriptionEngine -n sslKeyringFile -v "" 

mqsichangeproperties BRKR_SCCS -e default -o DynamicSubscriptionEngine -n sslPassphraseFile -v "" 

Enabling SSL
NOTE: Make sure these commands go on 1 line

mqsichangeproperties broker service -e default -o DynamicSubscriptionEngine 

-n clientAuthProtocols -v SP 

mqsichangeproperties BRKR_SCCS  -e default -o DynamicSubscriptionEngine  

-n clientAuthProtocols -v SP 

mqsichangeproperties broker service -e default -o DynamicSubscriptionEngine 

-n sslKeyringFile -v "c:\Program Files\IBM\MQSI\6.0\.keystore" 

mqsichangeproperties broker service -e default -o DynamicSubscriptionEngine 

-n sslPassphraseFile -v "c:\Program Files\IBM\MQSI\6.0\.keypass" 

For more info to Enable SSL on the EventBroker see the Info Center for Advanced.

DB2 password has changed effects BROKER

When you change the DB2 password that is associated with the broker database, you have to update the brokers dbpassword. Try to change the ODBC Data Source (from control panel) password and restart broker. If this doesnt work then try changing the brokers password below.

C:\Program Files\IBM\MQSI\6.0>mqsichangebroker BRKR_SCCS  -p aramate1

BIP8071I: Successful command completion.

If it still does not work, you can delete the broker and configmgr, then run ConfigureEB.bat over.
To delete the broker & configmgr, run

mqsideletebroker BRKR_SCCS 

mqsideleteconfigmgr CMGR_SCCS

Where to get abend and dmp files

for Windows
On the Event Broker machine look in the 'C:\Documents and Settings\All Users\Application Data\IBM\MQSI\common\errors\' folder for ABEND/DMP files

How to look for CPU heap issues for Event Broker

See the following resources because when the broker runs out of memory this might cause the broker to produce an abend file

• Using dumps and abend files
• Checking for abend files

Set the JVM Heap MIN/MAX

mqsichangeproperties BRKR_SCCS -e default -o ComIbmJVMManager -n jvmMaxHeapSize -v 1610612736

jvmMaxHeapSize is the size of the Java™ Virtual Machine (JVM) heap size used with the JVMManager for your Java user-defined nodes.
This value must be in the range 16 777 216 (16MB) to 8 589 934 592 (8GB). 1610612736 bytes is equivalent to 1.5GB. An appropriate max is 512MB which is 536870912 bytes. For more info/usage see mqsichangeproperties command

mqsireportproperties BRKR_SCCS -e default -o DynamicSubscriptionEngine -r

How to monitor CPU for EB

There's a detailed IBM article on How to monitor system and WebSphere Business Integration Message Broker V5 resource use
which talks about using the Task Manager to monitor CPU cycles, memory, and I/O. You can bring up the Task Manager on any windows machine by CTRL+ALT+DEL and view CPU/Performance details. Find the process DataFlowEngine.exe (which is a WebSphere Business Integration Message Broker execution group) to see how much CPU/memory it is using.

Using netstat

*netstat to make sure that the correct ports are open

Go to Start - Run - type cmd - press enter. Now type netstat /a /o, where
-a Displays all connections and listening ports.
-o Displays the owning process ID associated with each connection.

• First you need to know the PID (Process ID that the Event Broker is using. Start the Task Manager (hit CTRL+ALT+DEL on your keyboard). In the Task Manager, go to View
   
  Set Columns, check PID (Process Identifier)
   
  hit OK. In the Task Manager, sort by the "Image Name" column by clicking on that tab. Look for DataFlowEngine.exe and see what PID it is associated with. Now when you run netstat, you should see the owning process ID associated with each connection. The PID that was used by DataFlowEngine should be present in a few of the connections shown by netstat. Look to see if the connection with port '1506' has status LISTENING which is usually a good sign. If no connections exist for your Event Broker PID, then try restarting the broker.
• Look for the port 1414 which is the WebSphere MQ port #.
• If you are doing clustering than port 1507 must be used. If the port is not open then it is not clustering.

Run the following to get all connections

>netstat -a -o 

Using linux

Use the command netstat -an with grep to find a particular port. You can see below examples show both ports to be "listening" which is expected.

Example:

# netstat -an | grep 1506

tcp        0      0 :::1506                 :::*                    LISTEN

tcp        0      0 9.33.30.83:1506         9.65.34.63:2118         ESTABLISHED

# netstat -an | grep 1414

tcp        0      0 :::1414                 :::*                    LISTEN

unix  3      [ ]         STREAM     CONNECTED     11414

Using the Information Center Event Broker

Starting and Stopping WebSphere MQ and WebSphere Event Broker Troubleshooting Event Broker password changes Broker or execution group startup trace Tips for installing DB2, Message Queue, and Event Broker